Analysis of the security and privacy of smart personal assistants with real and synthetic voices

C. Palacios-Castrillo, R. Palacios, R. Gesteira-Miñarro, A. Chávez-Macías, G. López

Journal of Information Security and Applications Vol. 101, pp. 104554

Resumen:

Smart Personal Assistants (SPA) can be trained with the owner's voice, and its voice features act as a biometric access password. The aim of this work was to analyze what information different personal assistants reveal without verifying the owner's voice, and what real risks exist in impersonating the owner's voice. To do this, a test protocol was defined, including commands for demanding generic information, personal information, and more sensitive requests such as making calls or purchases. To deceive the personal assistants, tests were carried out with various synthetic voices, including generative AI systems to create voice models based on the user registered in the assistants, hence allowing commands to be synthetically generated with the person's voice features. This study worked with Apple HomePod, Amazon Alexa, and Google Home assistants, which are the main devices on the market. It was possible to verify what type of information each system communicates without performing user validation and how accurate was the voice verification algorithm (activation command) depending on the synthetic voices used. We proposed a Synthetic Speech Detection system as a secondary security layer to identify whether a voice mimicking a target individual was synthetically generated. To evaluate this, a preliminary study on the fidelity of modern synthetic voices was conducted through subjective listening tests. The results indicate that human participants attained only a marginal performance above the 50% stochastic baseline, confirming the high perceptual transparency of current models and the inherent difficulty of the detection task.

Resumen divulgativo:

Este estudio analiza si asistentes inteligentes pueden ser engañados con voces sintéticas creadas por IA. Compara Apple, Amazon y Google, identifica riesgos para la privacidad y propone un sistema para detectar voces artificiales y reforzar la seguridad.

Palabras Clave: Privacy; Generative AI; Voice cloning; Smart personal assistant; Cybersecurity; DeepFake voices

Índice de impacto JCR-JIF y cuartil WoS: 4,400 - Q2 (2025)

Referencia DOI: https://doi.org/10.1016/j.jisa.2026.104554

Publicado en papel: Septiembre 2026.

Publicado on-line: Junio 2026.

Cita:
C. Palacios-Castrillo, R. Palacios, R. Gesteira-Miñarro, A. Chávez-Macías, G. López, "Analysis of the security and privacy of smart personal assistants with real and synthetic voices", Journal of Information Security and Applications, Vol. 101, pp. 104554, Septiembre 2026. [Online: Junio 2026] doi: 10.1016/j.jisa.2026.104554